Real-time aliasing rendering method for 3d vr video and virtual three-dimensional scene

ABSTRACT

Provided is a real-time aliasing rendering method for a 3D VR video and a virtual three-dimensional scene, including: capturing 3D camera video signals in real time and process the same to generate texture data; creating a virtual three-dimensional scene according to the proportion of a real scene; generating virtual camera rendering parameters according to a physical position of the 3D camera and a shooting angle relationship; aliasing the texture data onto a virtual three-dimensional object in a virtual scene, and adjusting the position of the virtual three-dimensional object according to a physical positional relationship between the virtual three-dimensional scene and the real scene, so as to form a complete virtual reality combined three-dimensional scene; rendering the virtual reality combined three-dimensional scene by using the virtual camera rendering parameters to obtain a simulated rendering picture.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority to PCT Application No.PCT/CN2017/100412, having a filing date of Sep. 4, 2017, the entirecontents of which is hereby incorporated by reference.

FIELD OF TECHNOLOGY

The following relates to the field of computer graphics imageprocessing, and in particular to a real-time aliasing rendering methodfor 3D VR video and virtual three-dimensional scene.

BACKGROUND

VR is the English abbreviation of Virtual Reality, and Chinese meansvirtual reality. VR video is a panoramic video mode rendered using VRtechnology. As the emerging video mode in the past two years, it hasdeveloped rapidly. It has become an important means of communication forthe development of multiple media such as print, television, movies, andthe Internet. It is also a commercial market that various software andhardware manufacturers focus on. But with the products on the markettoday, only simple VR video shooting and simple live broadcast functionscan be completed, and slightly more complicated effects require a lot ofcomplicated post-production work.

Currently in the product market, most of them are only focused on theresearch and development of camera equipment (panoramic cameras) andviewing equipment (VR glasses), but the investment in intermediateproduction equipment and three-dimensional graphics engine is verysmall, only foreign A few companies can provide similar products, butalso have problems such as single function and complicated operation.Among them, the rendering of video and graphic images is only a simplesuperimposed rendering of 3D video and graphic images, and does not havethe effect of simulation. The generally used 360-camera renderingvirtual camera does not adopt the method of simulating left and righteyes, but through subsequent re-production, the rendered video is simplyoutput to virtual equipment such as VR glasses to simulate the scene ofleft and right eye viewing. And in reality, the objects and scenes seenby the left and right eyes have different perspectives. Therefore, thevideo produced by the existing 3D video rendering technology lacksrealism in detail performance, no real immersion, and can be viewed fora long time. Can cause eye discomfort and dizziness.

SUMMARY

An aspect relates to a real-time aliasing rendering method for 3D VRvideo and virtual three-dimensional scenes. The method includes:

Collect 3D camera video signals in real time and process the videosignals to generate texture data;

Create a virtual 3D scene based on the real scene scale;

Generate and adjust virtual camera rendering parameters based on the 3Dcamera physical position and shooting angle relationship;

Overlaying the texture data on a specific virtual three-dimensionalobject in the virtual scene in the form of a texture map according tothe picture effect captured by the 3D camera, and adjusting the positionof the virtual three-dimensional object according to the physicalposition relationship between the virtual three-dimensional scene andthe real scene, Forming a three-dimensional scene combined with virtualreality;

The rendering process is adjusted corresponding to the renderingparameters of the virtual camera, and the virtual camera renderingparameters are used to render the three-dimensional scene combiningvirtual and reality to obtain a simulated rendering picture.

Optionally, the texture data includes left-eye scene texture data andright-eye scene texture data, and the left-eye scene texture data andright-eye scene texture data correspond to the left and right lenses ofthe 3D camera, respectively, as the left eye of the simulated human eyeScene and right eye scene.

In the process of aliasing the texture data to a specific virtualthree-dimensional object in the virtual scene, the left-eye scenetexture data is used when the left-eye scene is aliased, and theright-eye scene texture data is used when the right-eye scene isaliased. To form three-dimensional scene data of left and right eyescombined with virtual reality.

The process of using the virtual camera rendering parameters to renderthe three-dimensional scene data of the left and right eyes combinedwith virtual reality is further: when the left camera parameters of thevirtual camera are used to render the three-dimensional scene data ofthe left eye, the texture map uses the left lens of the virtual camera.Data; when rendering the 3D scene data of the right eye using the rightcamera parameters of the virtual camera, the texture map uses the dataof the right camera of the virtual camera.

The data of a set of simulated rendering images of the left and righteyes is stored in a set of the left and right eyes.

Optionally, the processing of the video signal includes deinterleavingprocessing of interlaced data and keying processing of the video sourcedata with a blue-green background.

In the step of creating a virtual 3D scene based on the real scenescale, you can optionally create a virtual 3D scene with a fixedconversion scale.

The conversion ratio between the virtual three-dimensional scene and thereal scene may be 100: 1.

Optionally, generating the virtual camera related parameters includes:camera position generation virtual camera position to simulate a realcamera position, camera rotation to simulate a real camera shootingangle, camera opening angle to simulate a real camera Zoom, and lensdistance to simulate a real camera lens distance.

When rendering the left-eye scene video signal according to the virtualcamera left lens parameter, the texture map uses the data correspondingto the left lens of the virtual camera; when rendering the right-eyescene video signal according to the virtual camera right lens parameter,the texture map uses the corresponding virtual camera right Lens data.

Optionally, rendering includes 360-degree panoramic rendering and VRglasses rendering.

Among them, 360-degree panoramic rendering typesets the left and righteye renderings of each group in an up-to-down 1: 1 layout and combinesthem into a complete picture up and down; VR glasses rendering typesetsthe left and right eye renderings of each group in a left-to-right 1: 1layout and combines them into a completed picture.

According to embodiments of the present invention, the 360 panoramicpictures of the left eye and the right eye and the virtual graphics andimages of the virtual three-dimensional scene are respectively renderedusing the position and perspective relationship of the simulated humaneye for two independent 360 panoramic renderings, and VR is performed byadopting the technology of the simulation principle of the human eyeperspective The rendering of perspective scenes guarantees that theoutput effect is close to the perspective perceived by the human eye, sothat the user is immersed in the environment, so that when the userexperiences, the left and right eyes can see the rendered images withpoor perspectives more realistically, and long-term viewing will notproduce eyes Discomfort, which greatly improves immersion and comfort.

BRIEF DESCRIPTION

Some of the embodiments will be described in detail, with references tothe following Figures, wherein like designations denote like members,wherein:

FIG. 1 is a flowchart of a method for real-time aliasing rendering of a3D VR video and a virtual 3D scene according to embodiments of thepresent invention;

FIG. 2 is a flowchart of a method for real-time aliasing rendering of a3D VR video and a virtual 3D scene according to an embodiment of thepresent invention. In this embodiment, left and right eye simulationsare performed through left and right lenses of a 3D camera;

FIG. 3 is a schematic layout diagram of the VR glasses typesetting inFIG. 2 according to an embodiment of the present invention, whichsimulates left and right eyes through 3D camera left and right lenses;and

FIG. 4 is a schematic diagram of 360 panoramic typesetting of left andright eye simulation through left and right lenses of a 3D cameraaccording to an embodiment of the present invention.

DETAILED DESCRIPTION

In order to make the objectives, technical solutions, and advantages ofembodiments of the present invention clearer, embodiments of the presentinvention will be described in further detail with reference to theaccompanying drawings. The embodiments described below are onlyembodiments of the present invention, and they are only used to makeembodiments of the present invention more clear. Explanation anddescription are not used to limit the protection scope of embodiments ofthe present invention.

Referring to FIG. 1, FIG. 1 is a flowchart of a method for real-timealiasing rendering of a 3D VR video and a virtual 3D scene.

As shown in FIG. 1, the method includes:

S101. Collect and process a 3D camera signal.

In this step, a 3D camera video signal is collected and processed inreal time to generate texture data;

The processing of the video signal may include deinterleaving processingof interlaced data and keying processing of the video source data with ablue-green background.

S102. Create a virtual three-dimensional scene.

Create a virtual 3D scene according to the real scene scale; optionally,create a virtual 3D scene with a fixed conversion scale;

S103. Adjust parameters related to the virtual camera.

Generate virtual camera rendering parameters according to the real 3Dcamera physical position and shooting angle relationship;

Optionally, generating the virtual camera related parameters includes:camera position generation virtual camera position to simulate a realcamera position, camera rotation to simulate a real camera shootingangle, camera opening angle to simulate a real camera Zoom, and lensdistance to simulate a real camera lens distance;

S104. Aliasing forms a three-dimensional scene combined with virtualreality.

Overlaying the texture data on a specific virtual three-dimensionalobject in the virtual scene in the form of a texture map according tothe picture effect captured by the 3D camera, and adjusting the positionof the virtual three-dimensional object according to the physicalposition relationship between the virtual three-dimensional scene andthe real scene, Form a three-dimensional scene combined with virtualreality.

S105. Render a three-dimensional scene combined with virtual reality.

The rendering process is adjusted corresponding to the renderingparameters of the virtual camera, and the virtual camera renderingparameters are used to render the three-dimensional scene combiningvirtual and reality to obtain a simulated rendering picture.

Referring to FIG. 2, FIG. 2 is a flowchart of another embodiment of amethod for real-time overlay rendering of a 3D VR video and a virtual 3Dscene in FIG. 1.

In this embodiment, the human eyes are simulated through the left andright lenses of the 3D camera. The specific steps are:

S201: Collect and process the left and right lens scene signals of the3D camera in real time to generate texture data for the left eye sceneand texture data for the right eye scene.

The scene signals captured by the left and right lenses of the 3D cameraare used as the left-eye scene signal and the right-eye scene signal,respectively, for simulating the left-eye scene and the right-eye sceneof the human eye.

The processing of the video signal may include deinterleaving processingof interlaced data and keying processing of the video source data with ablue-green background.

S202. Create a virtual three-dimensional scene.

Create a virtual three-dimensional scene according to the real scenescale; optionally, create a virtual three-dimensional scene with a fixedconversion scale.

S203: Adjust parameters related to the virtual camera.

Generate virtual camera rendering parameters based on the real 3D cameraphysical position and shooting angle relationship.

Optionally, generating the virtual camera related parameters includes:camera position generation virtual camera position to simulate realcamera position, camera rotation to simulate real camera shooting angle,camera opening angle to simulate real camera Zoom, and lens distance tosimulate real camera lens distance.

S204. Aliasing forms a three-dimensional scene combined with virtualreality. According to the picture effect captured by the 3D camera, thetexture data is aliased onto a specific virtual three-dimensional objectin the virtual scene. Specifically, according to the picture effectcaptured by the 3D camera, the texture data is aliased into the form ofa texture map. On the specific virtual 3D object in the virtual scene,and adjust the position of the virtual 3D object according to thephysical position relationship between the virtual 3D scene and the realscene to form a 3D scene combined with virtual reality.

Wherein, the left-eye scene texture data is used when the left-eye sceneis aliased, and the right-eye scene texture data is used when theright-eye scene is aliased to form three-dimensional scene data of theleft and right eyes combined with virtual reality.

S205. Use virtual camera rendering parameters to render the left andright eye three-dimensional scene data of the virtual and realitycombination.

Among them, when rendering the left-eye 3D scene data according to theleft lens parameter of the virtual camera, the texture map uses the dataof the left lens of the virtual camera; when rendering the 3D scene dataof the right eye according to the right lens parameter of the virtualcamera, the texture map uses the right of the virtual camera Lens data.

In addition, the simulated 3D rendered picture data can be stored in agroup of left and right eyes.

Referring to FIG. 3, in the manner of VR glasses typesetting for leftand right eye simulation through the left and right lenses of the 3Dcamera, the scenes taken by the left and right lenses of the 3D cameraare respectively left eye scene and right eye scene, and are collectedand processed to generate multiple The left-eye scene texture data andthe right-eye scene texture data of the group are simulated for the leftand right eyes.

In this mode, rendering may include 360-degree panoramic rendering andVR glasses rendering.

Among them, 360-degree panoramic rendering typesets the left and righteye renderings of each group in an up-to-down 1: 1 layout and combinesthem into a complete picture.

Referring to FIG. 4, in a 360 panorama typesetting method for left andright eye simulation through the left and right lenses of the 3D camera,the scenes captured by the left and right lenses of the 3D camera arerespectively used as the left eye scene and the right eye scene, and arecollected and processed to generate multiple groups The left-eye scenetexture data and the right-eye scene texture data are simulated for theleft and right eyes.

In this mode, the VR glasses rendering is to typeset the left and righteye renderings of each group in a left-to-right 1: 1 layout and finallycombine them into a complete picture.

It can be seen from the above embodiments that the present inventionperforms independent two 360 panorama renderings by using the 360-degreepanoramic pictures of the left and right eyes and the virtual graphicsand images of the virtual three-dimensional scene to simulate theposition and perspective relationship of the human eye, respectively.When the user experiences, the left and right eyes respectively see therendered pictures with poor perspectives, which is more realistic andmore immersive.

Although the invention has been illustrated and described in greaterdetail with reference to the preferred exemplary embodiment, theinvention is not limited to the examples disclosed, and furthervariations can be inferred by a person skilled in the art, withoutdeparting from the scope of protection of the invention.

For the sake of clarity, it is to be understood that the use of “a” or“an” throughout this application does not exclude a plurality, and“comprising” does not exclude other steps or elements.

1. A real-time aliasing rendering method for three-dimensional virtualreality (3D VR) video and a virtual three-dimensional scene, the methodcomprising: acquiring in real-time a video signal of a 3D camera andprocessing of the video signal to generate texture data; creating avirtual three-dimensional scene according to a real scene ratio;adjusting the-rendering parameters of a virtual camera according to arelationship between a physical position of the 3D camera and the ashooting angle; according to a picture effect captured by the 3D camera,aliasing the texture data in a form of texture maps to specific virtualthree-dimensional objects in the virtual three-dimensional scene, andadjusting a position of the virtual three-dimensional object by aphysical position relationship between the virtual three-dimensionalscene and the real scene to form a virtual and reality combinedthree-dimensional scene; and adjusting the rendering processcorresponding to the virtual camera rendering parameters, and using thevirtual camera rendering parameters to render the virtual and realitycombined three-dimensional scene to obtain the simulation rendering. 2.The real-time aliasing rendering method for 3D VR video and a virtualthree-dimensional scene according to claim 1, wherein the texture dataincludes left-eye scene texture data and right-eye scene texture data,the left-eye scene texture data and the right eye scene texture datacorresponds to the left and right lenses of the 3D camera respectively,as the left-eye scene and the right-eye scene of the simulated humaneye.
 3. The real-time aliasing rendering method for 3D VR video and avirtual three-dimensional scene according to claim 2, wherein in theprocess of aliasing the texture data onto a specific virtualthree-dimensional object in the virtual scene, the left-eye scenetexture data is used when the left-eye scene is aliased, and theright-eye scene texture data is used when the right-eye scene isaliased, so as to form a virtual and reality combined three-dimensionalscene data of the left and right eyes.
 4. The real-time aliasingrendering method for 3D VR video and a virtual three-dimensional sceneaccording to claim 3, wherein the process of rendering the 3D scene dataof the left and right eyes of the virtual and reality combination byusing virtual camera rendering parameters is further based on: when theleft camera parameter of the virtual camera is used to render the lefteye 3D scene data, the texture map uses the data of the left lens of thevirtual camera; when the right camera parameter of the virtual camera isused to render the right eye 3D scene data, the texture map uses thedata of the right camera's virtual camera .
 5. The real-time aliasingrendering method for 3D VR video and a virtual three-dimensional sceneaccording to claim 1, wherein the video signal processing includesdeinterlacing the interlaced data and performing keying to theblue-green background of the video source data.
 6. The real-timealiasing rendering method for 3D VR video and a virtualthree-dimensional scene according to claim 1, wherein the virtualthree-dimensional scene is created by using a fixed conversion ratio. 7.The real-time aliasing rendering method for 3D VR video and a virtualthree-dimensional scene according to claim 1, wherein generating thevirtual camera-related parameters includes: camera position generatesvirtual camera position to simulate real camera position, and camerarotates to simulate real camera shooting angle and camera opening anglesimulates the real camera Zoom and the lens distance simulates the realcamera lens distance.
 8. The real-time aliasing rendering method for 3DVR video and a virtual three-dimensional scene according to claim 1,wherein the simulated 3D rendering picture data is stored in a group ofleft and right eyes
 9. The real-time aliasing rendering method for 3D VRvideo and a virtual three-dimensional scene according to claim 1,wherein the rendering comprises 360-degree panoramic rendering and VRglasses rendering.
 10. The real-time aliasing rendering method for 3D VRvideo and a virtual three-dimensional scene according to claim 1,wherein the 360-degree panoramic rendering typesets the left and righteye rendering pictures of each group in an up-to-down 1:1 layout andcombines them into a complete picture; and VR glasses rendering typesetsthe left and right eye rendering pictures of each group in aleft-to-right 1:1 layout and combines them into a complete picture.